1. Field of the Invention.
The invention is in the field of image processing performed on digital images, more particular on medical images such as mammographic images.
More particularly the invention relates to the automatic detection of objects or structures which have physical significance within a complex or cluttered digital image, such as a digitised medical image (e.g. a digital mammogram used for detecting breast cancer, or else a film-based mammogram which has been digitised so as to be suitable for analysis). In the case of mammography, the objects to be detected are radiological opacities which might be considered by radiologists to be suspiciously abnormal. More generally, the objects could be any suspicious radiological opacity (e.g. in chest X-rays) or could be a discrete feature in any other form of digital imaging (e.g. an object of interest in a sonar scan).
As indicated above, the invention has a wide range of potential application. The invention has been made within the specific context of breast imaging, and in particular as part of an attempt to construct a system which can automatically detect suspicious features within a breast image and alert a radiologist to the presence of those features. Radiologists are interested in a number of different abnormal features: this invention relates to the detection specifically of radiological opacities, which can include radiological categories such as ill-defined masses and stellate or spiculated masses or lesions.
Such an invention functions by obtaining a representation of a scene or image which can be manipulated and transformed by a computer system. The invention thus comprises an image acquisition part, a computation part, and an output part which either presents results of the computer manipulation or passes those results onto a further computer system for further analysis. The methods applied within the computer subsystem are particularly critical to the success of the overall system.
The following description relates to the computer""s manipulations.
2. Description of the Prior Art
For the purposes of this description we shall define an image to be a representation of a physical scene, in which the image has been generated by some imaging technology: examples of imaging technology could include television or CCD cameras or X-ray, sonar or ultrasound imaging devices. The initial medium on which an image is recorded could be an electronic solid-state device, a photographic film, or some other device such as a photostimulable phosphor. That recorded image could then be converted into digital form by a combination of electronic (as in the case of a CCD signal) or mechanical/optical means (as in the case of digitising a photographic film or digitising the data from a photostimulable phosphor). The number of dimensions which an image could have could be one (e.g. acoustic signals), two (e.g. X-ray radiological images) or more (e.g. nuclear magnetic resonance images).
The general problem to be tackled is to analyse such an image so as to identify, or help identify, the brightness fluctuations in the image which result from the presence of discrete physical structures within the original scene. This is relatively easy in cases where an image comprises only a uniform background brightness with a single localised fluctuation in brightness caused by an object, or where an image contains brightness fluctuations due to more than one discrete object but which are widely separated on the image. The aim of this invention is to help analyse more complex images, where the brightness fluctuations due to the object of interest are superimposed on a background of variable brightness, or where a number of objects are either physically close together or even overlapping. The latter problem is particularly acute in images which are two-dimensional projections of a three-dimensional scene, such as radiological images, and we shall tend to refer to the latter in the subsequent discussion. We shall refer to this type of image as being xe2x80x9cclutteredxe2x80x9d.
A typical example of a cluttered image, and one for which this invention was specifically targeted, is mammographic images, in which the normal structure within a breast creates a complex background of structures, which have highly variable X-ray opacity, size and shape and which frequently overlap and intersect.
A powerful approach to helping to separate out these various structures has become known as multiresolution or multiscale analysis. The aim is to segregate structures in an image into ranges of sizes. In that way, structures in the size-range of interest can be enhanced or individually identified. This is an active research field, and a number of techniques have been proposed and used for carrying out multiscale analysis.
Multiresolution Analysis and Multiresolution Pyramids
The principle usually applied to multiresolution analysis is to smooth, or convolve, an image with a chosen function. If the smoothing function is a low-pass filter, this has the effect of supressing or eliminating high-frequency Fourier components. A set of multiresolution images can be constructed by smoothing the image with a succession of such functions, whose size increases by some factor (usually a factor 2) between each step. The differences between successive steps, or resolution levels, contain information on structures whose size are approximately in the range between the sizes of the filters that were used to generate the difference image. In general, we should expect that lower-resolution data can be represented by a coarser rate of sampling, in which case we would have constructed a multiresolution pyramid (e.a. Burt P J and Adelson E H. 1983. IEEE Transactions on Communications. 31, 532-540).
Wavelet Analysis
An improved approach to generating a multiresolution analysis or pyramid has become known as wavelet analysis (Mallat S G. 1989a. IEEE Transactions on Pattern Analysis and Machine Intelligence. 11, 674-693 and Mallat SG. 1989b, Transactions of the American Mathematical Society, 315. 6-87, Daubechies I., 1992. xe2x80x9cTen Lectures on Waveletsxe2x80x9d, CBMS-NSF Regional Conference Series in Applied Mathematics. Society for Industrial and Applied Mathematics, Philadelphia, Pa. Chui C K, 1992 (ed). xe2x80x9cWavelet Analysis and its Applicationsxe2x80x9d, Vol. I and II, Academic Press), in which the smoothing functions that link each resolution level are carefully chosen to achieve optimum results for the application. If the smoothing function is chosen so that it obeys a scaling law.       φ    ⁢          (      x      )        =      2    ⁢                  ∑        n            ⁢                        h          n                ⁢                  φ          ⁢                      (                                          2                ⁢                x                            -              n                        )                              
then a multiresolution pyramid may be constructed such that no information is lost in constructing the pyramid. The difference images between successive resolution levels may themselves be encoded as a set of sample points to be convolved with a set of basis functions which are translates of a function known as the wavelet. The total number of sample points in the resulting pyramid may be no more than the number of sample points in the original imagexe2x80x94a useful feature where data volume may he a considerationxe2x80x94and the process may be inverted, starting with the multiresolution pyramid to reconstruct exactly the original data.
Having produced the wavelet coefficients for an image, they may then be used to create a lossy compressed image in data-compression applications, or to analyse the structures in an image by measuring the amplitude of the wavelet coefficients at resolution levels of particular interest.
Examples in mammography include the detection of microcalcifications (e.g. Clarke L P. Kallergi M, Qian W. Li H-D, Clark R A and Silbiger M L. 1994. Cancer Letters. 77, 173-181xe2x80x94Laine A F Schuler S. Fan J and Huda, 1994. IEEE Translation on Medical Imaging, 13, 725-740) or in providing a method of texture analysis which forms part of a method for detecting abnormal masses (e.g. Wei D. Chan H-P Helvie M A, Sahiner B. Petrick N. Adler D D and Goodsitt M M, 1995. Medical Physics. 22, 1501-1513).
Disadvantages of Wavelet Analysis
The previous approach using wavelets is not perfect, partly because the wavelet analysis does not perfectly segregate objects of different sizes. It is not usually possible to choose bases of wavelets and scaling functions which are both orthonormal and whose scaling function is an accurate representation of the structure of real objects in an image: this is because orthonormal basis functions must have negative components, which is often unphysical, and they cannot be symmetric and have compact support (e.g. Daubechies I., 1992, xe2x80x9cTen Lectures on Waveletsxe2x80x9d, CBMS-NSF Regional Conference Series in Applied Mathematics, Society for Industrial and Applied Mathematics, Philadelphia, Pa.). If the objects in an image to be analysed have arbitrary intensity profiles and arbitrary two-dimensional shapes it is not possible to choose basis functions that will create wavelet coefficients only at the resolution level of interest. More appropriate scaling functions can be chosen (e.g. B-spline scaling functions) if the orthonormality constraint is relaxed into a biorthogonality constraint, but even then the scaling functions will not match closely real physical objects. Furthermore, a low-resolution level in the multiresolution pyramid cannot contain the information on the high-resolution positioning of an object, and hence small-scale wavelet coefficients are needed to encode that information.
Even more critically, when using wavelet analysis to analyze the scale-size structure of images, since the wavelets coefficients are generated by a high-pass filter, the summed signal in wavelet coefficients generated by any one object at any one level is always zero: thus although wavelets can be used to locally enhance a region containing an object of a particular size, they cannot be used to completely remove that object from the image without using the wavelet coefficients covering all resolution levels. The wavelet (and its dual in the case of biorthogoanl analysis) are missing the vital low-frequency components which are necessary to achieve correct scale-size segregation of objects: wavelet analysis cannot create a representation of an individual object as it would appear had it been viewed in isolation, rather than in the complex image it actually occurred within.
Median and Morohological Filtering
Other means of analysing the scale-size information have been proposed using filters which are spatially non-linear. These include median and rank-order filters (e.g. Clarke L P, Kallergi M, Qian W. Li H-D, Clark R A and Silbiger M L. 1994. Cancer Letters, 77, 173-181) and mathematical morphology (e.g. Serra J. xe2x80x9cImage Analysis by Mathematical Morphologyxe2x80x9d, Academic Press, 1982). In one dimension such filters can be constructed which have some useful properties, but again the complete reconstruction of objects using a restricted portion of a scale-size anaylysis is not possible, and in two or more dimensions even the edge-preserving properties which make median filtering attractive are lost.
The problem with all the above analyses is that none of them attempt to estimate the amplitude and phase of the low-frequency Fourier components which are associated with an object. To do so is an inverse problem which does not have an unique solution, and the aim of the new work is to find an optimum method of estimating the amplitudes and phases of the Fourier components so that a complete reconstruction of individual objects may be made.
It is an object of the present invention to provide a method of detecting automatically objects or structures which have physical significance within a complex or cluttered digital image.
It is a further object to provide such a method that is particularly suitable for the automatic detection of such objects or structures in a digitised medical image. e.g. a digital mammogram.
A particular object of this invention is to find an optimum method of estimating the amplitudes and phases of the Fourier components associated with an object so that a complete reconstruction of individual objects may be made.
Still further objects will become apparent from the description hereafter.